Learning with Average Top-k Loss
نویسندگان
چکیده
In this work, we introduce the average top-k (ATk) loss as a new ensemble loss for supervised learning, which is the average over the k largest individual losses over a training dataset. We show that the ATk loss is a natural generalization of the two widely used ensemble losses, namely the average loss and the maximum loss, but can combines their advantages and mitigate their drawbacks to better adapt to different data distributions. Furthermore, it remains a convex function over all individual losses, which can lead to convex optimization problems that can be solved effectively with conventional gradient-based method. We provide an intuitive interpretation of the ATk loss based on its equivalent effect on the continuous individual loss functions, suggesting that it can reduce the penalty on correctly classified data. We further give a learning theory analysis of MATk learning on the classification calibration of the ATk loss and the error bounds of ATk-SVM. We demonstrate the applicability of minimum average top-k learning for binary classification and regression using synthetic and real datasets.
منابع مشابه
Smooth Loss Functions for Deep Top-k Classification
The top-k error is a common measure of performance in machine learning and computer vision. In practice, top-k classification is typically performed with deep neural networks trained with the cross-entropy loss. Theoretical results indeed suggest that cross-entropy is an optimal learning objective for such a task in the limit of infinite data. In the context of limited and noisy data however, t...
متن کاملEnsemble-based Top-k Recommender System Considering Incomplete Data
Recommender systems have been widely used in e-commerce applications. They are a subclass of information filtering system, used to either predict whether a user will prefer an item (prediction problem) or identify a set of k items that will be user-interest (Top-k recommendation problem). Demanding sufficient ratings to make robust predictions and suggesting qualified recommendations are two si...
متن کاملTop-k Consistency of Learning to Rank Methods
This paper is concerned with the consistency analysis on listwise ranking methods. Among various ranking methods, the listwise methods have competitive performances on benchmark datasets and are regarded as one of the state-of-the-art approaches. Most listwise ranking methods manage to optimize ranking on the whole list (permutation) of objects, however, in practical applications such as inform...
متن کاملAnalysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification
Top-k error is currently a popular performance measure on large scale image classification benchmarks such as ImageNet and Places. Despite its wide acceptance, our understanding of this metric is limited as most of the previous research is focused on its special case, the top-1 error. In this work, we explore two directions that shed light on the top-k error. First, we provide an in-depth analy...
متن کاملGreater Than Average: Optimizing a Convex Upper Bound on Top-K Loss
We consider the problem of class ambiguity which is widespread in scene recognition. We argue that the topK performance metric is more sensible in that domain and propose a novel loss function, which is a convex upper bound on the top-K error. Additionally, we compare performance of various loss functions in multiclass recognition and discover that multiclass losses perform significantly better...
متن کامل